Skip to content

Remove database usage from push notification extension #452

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 99 commits into
base: dev
Choose a base branch
from

Conversation

mpretty-cyro
Copy link
Collaborator

@mpretty-cyro mpretty-cyro commented May 13, 2025

The iOS app has had a long standing crash related to the database, the app crashes with the error code 0xdead10cc (or “Deadlock”) and it happens when the app goes into a “suspended” state whilst it still maintains a lock on the database. We've tried a number of times to resolve this issue without a large structural change but, while we seem to have reduced the frequency, we've been unable to entirely resolve it.

Since we have exhausted all other options the goal now is to move the database file (and any other app data) from the “AppGroup” into the main app directory where the OS will no longer need to put a lock on the file, and as a result the 0xdead10cc should no longer be possible.

The purpose of this PR is to update the Push Notification Extension to function without needing access to the database (the first step needed to relocate the app files).

Changes in this PR include:

  • Refactored the push notification extension to no longer need the database
  • Refactored identity verification and crypto usage to no longer need the database
  • Refactored message deduplication
  • Refactored logic to determine whether to show a notification so it doesn't rely on the database
  • Refactored message sending pipeline to no longer require database access
  • Refactored attachment upload process to no longer require database access
  • Refactored the OpenGroupAPI to no longer require database access
  • Refactored the SessionNetworkAPI to no longer require database access
  • Added logic to replicate required data from the database to disk so the PN extension has access to it
  • Updated the code to retrieve the current users profile from libSession instead of the database
  • Updated the code to store and retrieve user preferences with libSession instead of the database
  • Fixed a bug where we could be incorrectly stripping numbers from messages
  • Fixed an issue where the notification extension wasn't correctly reporting the successful handling of a push notification
  • Fixed a BencodeDecoder bug

mpretty-cyro and others added 2 commits May 12, 2025 14:36
• Updated the General cache to include the users seckey (rather than fetching it from the DB every time causing unneeded load)
• Updated a bunch of Crypto usage to no longer need a `db` or `dependencies` instance passed to it
• Added unit tests for General.Cache
• Replaced `Identity.userExists` with `General.Cache.userExists` (no need for database access anymore)
• Moved the SwarmPoller message handling logic into it's own function to make debugging stack traces a little easier
• Cleaned up some hard-to-read logic
DBR - 1 - Cached seckey, cleaned up crypto usage
@mpretty-cyro mpretty-cyro self-assigned this May 13, 2025
@mpretty-cyro mpretty-cyro added enhancement New feature or request Jira This ticket is being tracked in Jira labels May 13, 2025
mpretty-cyro and others added 25 commits May 13, 2025 14:03
• Added the new MessageDeduplication table
• Added a migration to populate the MessageDeduplication table
• Added ExtensionHelper functions to store records in the AppGroup for deduplication purposes
• Updated the logic to use MessageDeduplication table & files for deduplication purposes
• Refactored message parsing to not rely on the database
• Refactored message deduplication to not rely on the database in the PN extension
• Removed the old ControlMessageProcessRecord table
…receiving-process

DBR - 2 - Refactored the message deduplication logic
• Updated the General cache to include the users seckey (rather than fetching it from the DB every time causing unneeded load)
• Updated a bunch of Crypto usage to no longer need a `db` or `dependencies` instance passed to it
• Added unit tests for General.Cache
• Replaced `Identity.userExists` with `General.Cache.userExists` (no need for database access anymore)
• Moved the SwarmPoller message handling logic into it's own function to make debugging stack traces a little easier
• Cleaned up some hard-to-read logic
…message-notification

DBR - 3 - Initial notification extension refactoring, handling visible message PNs
• Added the new MessageDeduplication table
• Added a migration to populate the MessageDeduplication table
• Added ExtensionHelper functions to store records in the AppGroup for deduplication purposes
• Updated the logic to use MessageDeduplication table & files for deduplication purposes
• Refactored message parsing to not rely on the database
• Refactored message deduplication to not rely on the database in the PN extension
• Removed the old ControlMessageProcessRecord table
• Added code to write user metadata to disk
• Added code to write config dumps to disk
• Added code to write push notification messages to disk
• Added code to read push notification messages from disk on launch
• Added code to calculate the unread count in the PN extension (based on the last known main app unread count)
• Added code to replicate config dumps to the AppGroup for extensions to load
• Added async/await read & write functions to Storage
• Tweaked the logic when the user taps on a notification to wait until after the PN extension message files have been loaded (with a 3s timeout)
• Fixed an issue where notifications were being overwritten in all cases (now using the serverHash as the identifier when available)
• Added unit tests for new ExtensionHelper functions
• Tweaked the `#file` variables to use `#fileID` instead (relative to project dir)
• Updated AppDelegate so it doesn't run when running unit tests (could slow other stuff down or cause bugs)
• Fixed broken unit tests
• Fixed an issue where the notification extension wasn't correctly reporting the successful handling of a push notification
…ving-to-disk

DBR - 4 - Saving and reading messages from disk
# Conflicts:
#	Session.xcodeproj/project.pbxproj
#	SessionMessagingKit/Open Groups/OpenGroupManager.swift
#	SessionMessagingKit/Sending & Receiving/Notifications/NotificationsProtocol.swift
#	SessionMessagingKit/Sending & Receiving/Notifications/Types/NotificationCategory.swift
#	SessionMessagingKitTests/_TestUtilities/CommonSMKMockExtensions.swift
#	SessionMessagingKitTests/_TestUtilities/MockNotificationsManager.swift
#	SessionNotificationServiceExtension/NSENotificationPresenter.swift
#	SessionUtilitiesKit/General/SessionId.swift
• Updated the General cache to include the users seckey (rather than fetching it from the DB every time causing unneeded load)
• Updated a bunch of Crypto usage to no longer need a `db` or `dependencies` instance passed to it
• Added unit tests for General.Cache
• Replaced `Identity.userExists` with `General.Cache.userExists` (no need for database access anymore)
• Moved the SwarmPoller message handling logic into it's own function to make debugging stack traces a little easier
• Cleaned up some hard-to-read logic
• Added the new MessageDeduplication table
• Added a migration to populate the MessageDeduplication table
• Added ExtensionHelper functions to store records in the AppGroup for deduplication purposes
• Updated the logic to use MessageDeduplication table & files for deduplication purposes
• Refactored message parsing to not rely on the database
• Refactored message deduplication to not rely on the database in the PN extension
• Removed the old ControlMessageProcessRecord table
• Updated display pictures and attachments to use a hash derived from the download url instead of a random UUID for the filename
• Renamed some database columns for consistency
• Dropped a bunch of unused database columns
• Removed old quoted attachment thumbnail logic
• Fixed an issue where updating the display picture wouldn't update the settings UI
• Fixed an issue where image thumbnails may not render correctly
• Fixed an issue where some text wouldn't be respect the theme
• Fixed a performance issue with one of our list components
• Fixed an issue where sharing files wasn't saving the attachment data correctly
• Added better support for failing cases in the custom media transitions
• Tweaked a bunch of task priorities (and made a bunch detached tasks
• Fixed a couple of bugs with the custom media transitions
• Fixed an specific case where the conversation list could stop updating
• Fixed an issue where the share extension could log multiple times
• Fixed a rare thread two-lock deadlock issue
• Fixed an issue where we would incorrectly deduplicate config messages (libSession should handle that internally)
• Fixed an issue where group invites were not encrypting correctly
• Fixed an issue where mentioning a non-contact group member wouldn't show their name in the push notification
• Fixed an issue where group configs may not replicate correctly
• Fixed an issue where a title-only notification would show for the currently open conversation
• Fixed an issue where the first message request message wouldn't show a PN
• Cleaned up some async usage that was deadlock prone
• Fixed a deadlock that could occur in the PagedDatabaseObserver
• Fixed an issue where contact config merges could fail due to blinded conversations
• Fixed an issue where the GroupLeavingJob could fail
• Fixed an issue where the home screen might not update
• Reworked the async/await observation logic to be separate from libSession (resolve a complex deadlock)
…eature/database-relocation

# Conflicts:
#	Session/Conversations/ConversationVC+Interaction.swift
#	Session/Conversations/ConversationVC.swift
#	Session/Conversations/Settings/ThreadSettingsViewModel.swift
#	Session/Shared/Views/SessionCell+AccessoryView.swift
#	SessionMessagingKit/Database/Models/SessionThread.swift
#	SessionMessagingKit/Utilities/ProfilePictureView+Convenience.swift
#	SessionUIKit/Types/IconSize.swift
• Removed notification setting syncing
• Added logic to replicate local settings
• Fixed a bug the message deduplication migration was incorrectly adding expirations to community messages which shouldn't expire
• Fixed a bug where the GarbageCollectionJob would incorrectly remove dedupe records which shouldn't expire
• Fixed a bug where generic push notifications would incorrectly be shown for self-send messages
…ocation

# Conflicts:
#	Session.xcodeproj/project.pbxproj
• Added unit tests for the Onboarding Cache
• Updated the delete contact swipe icon to be consistent
• Made some optimisations to try to speed up the database migration unit tests
• Added code to prevent trying to load image data when the file is likely invalid
• Fixed a bug where the invite contacts list for a community wouldn't be populated
• Fixed a bug where profiles with downloadUrl values but no files wouldn't render the fallback display picture
• Fixed a bug where we were incorrectly suspending the database in the PN extension (we no longer use the database so this isn't needed)
• Fixed a bug where onboarding wasn't restoring the displayName correctly
• Fixed a bug where sending video attachments would cause a freeze
• Fixed a crash when changing a contacts nickname
• Rebuild the observation system so UI updates can be triggered arbitrarily rather than just from database changes (requires manual evert firing)
• Refactored the TypingIndicators to be an actor
• Fixed an issue where sharing something from within the app via Session wouldn't update the conversation list content
• Removed the "transientDependencies" and "currentlyObservedQueries" hacks
• Moved a bunch of "ThreadSafe" variables from the `Storage` type onto a TaskLocal `MigrationExecution.Context` value
• Cleaned up migration logic
• Tweaked some libSession error logs
• Added some more logs to the notification extension around group configs
• Updated a bunch of logic to be flagged as `@MainActor` to get compile level checks around thread safety
• Fixed a crash when joining a community
• Fixed an issue where some events wouldn't refresh data in an expected way
• Fixed an issue where the observation could be lost
• Updated the message requests screen to (mostly) use the same new observation system as the home screen
• Reworked the PagedData to be based on `ID` instead of `rowId` to remove a redundant query
• Fixed a modal styling issue
• Fixed an issue where showing/hiding the Note to Self conversation wouldn't update the home screen
• Fixed an issue where showing/hiding the Note to Self conversation on a linked device wouldn't update the home screen
• Fixed an issue where the unread count of a conversation wouldn't be updated unless the last message was read
• Fixed an issue where the group avatar might not be correctly generated
• Fixed an issue where the gradient fade behind footer buttons
• Fixed an issue where the fallback icon for the group avatar wasn't working
• Fixed an issue where sending/receiving a message for a conversation outside of the loaded page range wouldn't result in a conversation list update
• Fixed an issue where opening the app via a push notification when a screen was presented when backgrounding the app would result in the presented screen still being visible
• Removed the "users must be on the latest version" V2 groups banner
• Updated the NotificationSettingsViewModel to (mostly) use the new ObservationBuilder pattern
• Fixed a layout issue the with path status
• Fixed a copy discrepancy when deleting conversations from the swipe action
• Fixed an issue where an automated test couldn't see the empty state of the home screen correctly
• Fixed an issue where the message request banner wouldn't reappear when receiving a new message request after hiding it
• Fixed an issue where image thumbnails wouldn't load correctly
• Fixed an issue where the default notification sound value was incorrect
• Added a mechanism to trigger an immediate event (it forces the debouncer to flush)
• Updated the ThreadNotificationSettingsViewModel to use the ObservationBuilder
• Updated the AppearanceViewModel to use the ObservationBuilder
• Updated the ConversationSettingsViewModel to use the ObservationBuilder
• Updated the NotificationContentViewModel to use the ObservationBuilder
• Updated the NotificationSettingsViewModel to use the ObservationBuilder
• Updated the PrivacySettingsViewModel to use the ObservationBuilder
• Updated the SettingsViewModel to use the ObservationBuilder
• Cleaned up the MessageRequestsViewModel and HomeViewModel to be consistent with the other refactored view models
• Removed the `libSessionObservation` logic (replaced with new ObservationBuilder)
• Fixed a bug where the notification sound setting might not work correctly
• Fixed a few incorrect events being sent
• Attaching thread metadata to generic failure notification if available (will mean tapping on the notification opens the conversation)
• Updated the config dump replication process to be self-healing for missing dumps (on launch)
• Fixed an issue where the new account state would disappear too soon
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Jira This ticket is being tracked in Jira
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant